Lecture 1: August 27 1.1 the Markov Chain Monte Carlo Paradigm 1.2 Applications 1.2.1 Combinatorics

Markov Chain Monte Carlo constructs a Markov Chain (Xt) on Ω that converges to π, ie Pr[Xt = y|X0 = x]→ π(y) as t→∞, independent of x. Then we get a sampling algorithm by simulating the Markov chain, starting in an arbitrary state X0, for sufficiently many steps and outputting the final state Xt. It is usually not hard to set up a Markov chain that converges to the desired stationary distribution; however, the key question is how many steps is “sufficiently many,” or equivalently, how many steps are needed for the chain to get “close to” π. This is known as the “mixing time” of the chain. Obviously the mixing time determines the efficiency of the sampling algorithm.